Mobile camera localization involves finding the position and orientation of a camera moving in its environment and is useful for many applications such as robotics, immersive gaming, augmented reality, architecture, planning, robotics, engineering prototyping, vehicle navigation, medical applications and other problem domains. Existing approaches are limited in accuracy, robustness and speed. For many applications accurate camera localization is required in real time, for example, in order that a robot may successfully move about in its environment.
Many previous approaches to mobile camera localization have used color video cameras rather than depth cameras. Typically color video cameras give high resolution and accuracy and the rich color information allows visual features to be detected in the video images. Information available from depth cameras may be noisy and inaccurate depending on the type of environment and the type of depth camera used. Depth cameras capture images referred to herein as depth maps where each pixel relates to an absolute or relative distance from the depth camera to a point in the camera's environment. It may be harder to detect features in depth maps as compared with color images due to the difference in the information available.
Some previous approaches to mobile camera localization have involved creating a map of a mobile camera's environment at the same time as tracking the camera's position and orientation with respect to that map. This is known as simultaneous localization and mapping (SLAM).
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known mobile camera localization systems.